Occlusion-robust Visual Markerless Bone Tracking for Computer-Assisted Orthopaedic Surgery

Conventional computer-assisted orthopaedic navigation systems rely on the tracking of dedicated optical markers for patient poses, which makes surgical workflow more invasive, tedious, and expensive. To address this limitation, some previous studies have successfully adapted existing deep learning framework to automatically segment register exposed femur surface knee surgery, but these fail under real-world occlusion often happens during a real procedure. Furthermore, such methods are hardware-specific not accurate enough clinic acceptance. In paper, we propose learning-based RGB-D markerless method that is robust against occlusion. avoid expensive data collection, well-known challenge task training, generate synthetic with various scenarios. A new segmentation network features dynamic region-of-interest prediction 3D geometric designed, learn occlusion-related knowledge from simulated instances. Intensive experiments show our proposed achieves state-of-the-art results in bone tracking. generalises well cameras target models, including cadaver, without need retraining. By using high-quality camera, visual can achieve an accuracy 1-2° 2-4 mm phantom knee, meets clinical standard.